Do Autoencoders Need a Bottleneck for Anomaly Detection?

A common belief in designing deep autoencoders (AEs), a type of unsupervised neural network, is that bottleneck required to prevent learning the identity function. Learning function renders AEs useless for anomaly detection. In this work, we challenge limiting and investigate value non-bottlenecked AEs. The can be removed two ways: (1) overparameterising latent layer, (2) introducing skip connections. However, limited works have reported on use one ways. For first time, carry out extensive experiments covering various combinations removal schemes, types datasets. addition, propose infinitely-wide as an extreme example Their improvement over baseline implies not trivial previously assumed. Moreover, find architectures (highest AUROC=0.905) outperform their bottlenecked counterparts AUROC=0.714) recent benchmark CIFAR (inliers) vs SVHN (anomalies), among other tasks, shedding light potential developing improving